Skip to content

Conversation

@tac0turtle
Copy link
Contributor

Add header storage and retrieval to DummyDA to support timestamp determinism in tests. This enables tests to use the same header retrieval pattern as the real DA client.

Changes:

  • Add Header struct with Height, Timestamp, and Time() method
  • Store headers with timestamps when blobs are submitted
  • Store headers when height ticker advances
  • Add GetHeaderByHeight method mirroring HeaderAPI.GetByHeight
  • Update Retrieve to use stored timestamps for consistency

Closes #2944

🤖 Generated with Claude Code

Add header storage and retrieval to DummyDA to support timestamp
determinism in tests. This enables tests to use the same header
retrieval pattern as the real DA client.

Changes:
- Add Header struct with Height, Timestamp, and Time() method
- Store headers with timestamps when blobs are submitted
- Store headers when height ticker advances
- Add GetHeaderByHeight method mirroring HeaderAPI.GetByHeight
- Update Retrieve to use stored timestamps for consistency

Closes #2944

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@github-actions
Copy link
Contributor

github-actions bot commented Dec 20, 2025

The latest Buf updates on your PR. Results from workflow CI / buf-check (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed⏩ skipped✅ passed✅ passedJan 2, 2026, 11:16 AM

@claude
Copy link
Contributor

claude bot commented Dec 20, 2025

Claude finished @tac0turtle's task in 3m 13s —— View job


Comprehensive Code Review Complete ✓

I've performed a thorough code review of the header support implementation in DummyDA. The changes successfully add header storage and retrieval to support timestamp determinism in tests. Here's my detailed feedback:


Summary

Overall Assessment: The implementation is well-designed and addresses the stated goal of timestamp determinism. Previous review concerns have been properly addressed. I found a few areas for improvement around error handling consistency and documentation clarity.

Key Strengths:

  • Clean implementation that mirrors real DA client patterns (Celestia HeaderAPI)
  • Proper synchronization using mutexes for concurrent access
  • Headers are stored consistently in both Submit and StartHeightTicker
  • Good integration with e2e tests

Areas for Improvement: See detailed feedback below


Detailed Feedback

1. Code Quality & Architecture

test/testda/dummy.go:276-292 - GetHeaderByHeight error handling

The error handling in GetHeaderByHeight is inconsistent with Go best practices and the stated goal of determinism:

func (d *DummyDA) GetHeaderByHeight(_ context.Context, height uint64) (*Header, error) {
    d.mu.Lock()
    header := d.headers[height]
    d.mu.Unlock()

    if header == nil {
        currentHeight := d.height.Load()
        if height > currentHeight {
            return nil, datypes.ErrHeightFromFuture
        }
        return nil, datypes.ErrBlobNotFound  // ⚠️ Issue here
    }
    return header, nil
}

Issue: Returning ErrBlobNotFound when a header doesn't exist is semantically incorrect. This error name suggests a blob is missing, not a header. This could confuse callers and doesn't clearly communicate what went wrong.

Recommendation: Consider one of these approaches:

  1. Add a specific header error (preferred for clarity):
// In pkg/da/types/errors.go or appropriate location
var ErrHeaderNotFound = errors.New("header not found")

// Then in GetHeaderByHeight:
if header == nil {
    currentHeight := d.height.Load()
    if height > currentHeight {
        return nil, datypes.ErrHeightFromFuture
    }
    return nil, datypes.ErrHeaderNotFound
}
  1. Use ErrBlobNotFound consistently (if headers and blobs are conceptually equivalent):
    Add a comment explaining why this error is appropriate for headers too.

Impact: Medium - affects error handling clarity and debuggability


test/testda/dummy.go:135-140 - Retrieve timestamp handling

The fallback to zero-value timestamp is correct for determinism (unlike previous versions that used time.Now()), but could be documented better:

// Get timestamp from header if available, otherwise use current time
var timestamp time.Time
if header := d.headers[height]; header != nil {
    timestamp = header.Timestamp
}

Comment is outdated: The comment says "otherwise use current time" but the code actually uses zero-value time.Time{}, which is correct for determinism.

Recommendation: Update the comment:

// Get timestamp from header if available, otherwise use zero time for determinism
var timestamp time.Time
if header := d.headers[height]; header != nil {
    timestamp = header.Timestamp
}

Impact: Low - documentation clarity only


test/testda/dummy.go:107-113 - Header creation in Submit

The header creation logic is good, but there's a subtle race condition consideration:

d.blobs[height][nsKey] = append(d.blobs[height][nsKey], data...)
// Store header with timestamp for this height
if d.headers[height] == nil {
    d.headers[height] = &Header{
        Height:    height,
        Timestamp: now,
    }
}

Question: What happens if StartHeightTicker creates a header for this height before Submit does? The check if d.headers[height] == nil prevents overwriting, which is good for preserving the first timestamp, but this behavior isn't documented.

Recommendation: Add a comment explaining the first-timestamp-wins behavior:

// Store header with timestamp for this height (preserve first timestamp if already set)
if d.headers[height] == nil {
    d.headers[height] = &Header{
        Height:    height,
        Timestamp: now,
    }
}

Impact: Low - documentation clarity for maintainability


2. Error Handling & Resilience

test/testda/dummy.go:259-274 - Reset method

The Reset method properly clears headers, but there's a potential issue with concurrent access during reset:

func (d *DummyDA) Reset() {
    d.mu.Lock()
    d.height.Store(0)
    d.blobs = make(map[uint64]map[string][][]byte)
    d.headers = make(map[uint64]*Header)
    d.failSubmit.Store(false)
    d.mu.Unlock()

    d.tickerMu.Lock()
    if d.tickerStop != nil {
        close(d.tickerStop)
        d.tickerStop = nil
    }
    d.tickerMu.Unlock()
}

Concern: If GetHeaderByHeight is called after the first unlock but before the ticker is stopped, it could read from a cleared headers map while the ticker goroutine is still trying to write to it.

Recommendation: Consider stopping the ticker before clearing the headers:

func (d *DummyDA) Reset() {
    // Stop ticker first to prevent concurrent writes
    d.tickerMu.Lock()
    if d.tickerStop != nil {
        close(d.tickerStop)
        d.tickerStop = nil
    }
    d.tickerMu.Unlock()

    // Then clear state
    d.mu.Lock()
    d.height.Store(0)
    d.blobs = make(map[uint64]map[string][][]byte)
    d.headers = make(map[uint64]*Header)
    d.failSubmit.Store(false)
    d.mu.Unlock()
}

Impact: Low-Medium - race condition is unlikely in tests but could cause flaky test failures


3. Security

✅ No security concerns identified. This is test code with no production security requirements. The use of time.Now() for test timestamps is appropriate.


4. Performance & Resource Efficiency

test/testda/dummy.go:38 - Headers map

headers    map[uint64]*Header             // height -> header (with timestamp)

Observation: The headers map will grow unbounded in long-running tests. This is acceptable for test code but worth noting.

Consideration: For very long-running tests with many heights, this could consume memory. However, for typical test scenarios this is fine. The Reset() method allows cleanup when needed.

Impact: None for typical tests, informational only


5. Testing & Reliability

test/e2e/evm_test_common.go:494 - DA startup with header ticker

The e2e test setup now uses StartHeightTicker(1 * time.Second) which is good for creating headers at regular intervals. However:

Question: Should there be a test that specifically validates the header retrieval functionality? Currently, the e2e tests use the headers implicitly through timestamp checking, but there's no explicit test of GetHeaderByHeight.

Recommendation: Consider adding a focused unit test for GetHeaderByHeight in test/testda/dummy_test.go (if it doesn't exist) that validates:

  • Retrieving an existing header returns correct data
  • Retrieving a non-existent header returns appropriate error
  • Retrieving a future height returns ErrHeightFromFuture
  • Concurrent access patterns work correctly

Impact: Medium - would improve test coverage and confidence


6. Documentation & Developer Experience

test/testda/dummy.go:18-29 - Header struct documentation

The documentation is good but could be enhanced:

// Header contains DA layer header information for a given height.
// This mirrors the structure used by real DA clients like Celestia.
type Header struct {
	Height    uint64
	Timestamp time.Time
}

// Time returns the block time from the header.
// This mirrors the jsonrpc.Header.Time() method.
func (h *Header) Time() time.Time {
	return h.Timestamp
}

Suggestion: Add a brief explanation of why this mirrors Celestia's pattern:

// Header contains DA layer header information for a given height.
// This mirrors the structure used by real DA clients like Celestia,
// enabling tests to use the same header retrieval patterns as production code.
type Header struct {
	Height    uint64
	Timestamp time.Time
}

Impact: Low - improves understanding for future developers


test/testda/dummy.go:276-279 - GetHeaderByHeight documentation

The doc comment could be more comprehensive:

// GetHeaderByHeight retrieves the header for the given DA height.
// This mirrors the HeaderAPI.GetByHeight method from the real DA client.
// Returns nil if no header exists for the given height.

Suggestion: Document the error cases:

// GetHeaderByHeight retrieves the header for the given DA height.
// This mirrors the HeaderAPI.GetByHeight method from the real DA client.
//
// Returns:
//   - (*Header, nil) if header exists
//   - (nil, ErrHeightFromFuture) if height > current height
//   - (nil, ErrBlobNotFound) if height <= current height but no header exists
//
// Note: Headers are created when blobs are submitted or when the height
// ticker advances. Missing headers for valid heights indicate empty blocks.

Impact: Low - documentation clarity


Previous Review Feedback Status

I reviewed the previous comments from gemini-code-assist and alpe:

Gemini's concern about time.Now() fallbacks: This has been properly addressed. The current code in Retrieve (lines 135-140) uses zero-value time.Time instead of time.Now(), which maintains determinism as intended.

Gemini's concern about non-determinism in GetHeaderByHeight: The current implementation (lines 284-289) correctly returns errors rather than creating headers with time.Now(), maintaining determinism.

alpe's concerns: These reviews were dismissed, suggesting they've been addressed in subsequent iterations.


Test Changes Review

test/e2e/evm_full_node_e2e_test.go:494 - Minor change to add ticker:

sut.StartHeightTicker(1 * time.Second)

✅ Good addition - ensures headers are created at regular intervals for tests.

test/e2e/evm_test_common.go:494 - Same ticker addition:

✅ Consistent with the pattern across tests.


Recommendations Summary

High Priority: None - code is functionally correct

Medium Priority:

  1. Fix error semantics in GetHeaderByHeight - use ErrHeaderNotFound instead of ErrBlobNotFound for clarity
  2. Consider reordering operations in Reset() to stop ticker before clearing state
  3. Add unit tests specifically for GetHeaderByHeight functionality

Low Priority:

  1. Update outdated comment in Retrieve method (line 135)
  2. Document first-timestamp-wins behavior in Submit (line 108)
  3. Enhance documentation for Header struct and GetHeaderByHeight

Conclusion

This is a solid implementation that successfully achieves its goal of adding timestamp determinism to DummyDA tests. The code is well-structured, properly synchronized, and integrates cleanly with the existing codebase. The main improvements would be around error handling consistency and documentation clarity.

The previous review concerns about non-determinism have been properly addressed. All critical functionality works correctly, and the changes follow Go idioms and the project's coding standards.

Recommendation: Approve with minor suggestions for improvement.

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @tac0turtle, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the DummyDA test implementation by integrating header storage and retrieval capabilities. The core objective is to achieve timestamp determinism within test scenarios, enabling tests to interact with the DummyDA in a manner that closely mimics the behavior of a real Data Availability (DA) client's header API. This ensures that test environments accurately reflect production system behavior concerning block header information, particularly timestamps, leading to more robust and reliable testing.

Highlights

  • Header Struct Introduction: A new Header struct has been defined within DummyDA to encapsulate height and timestamp information, mirroring the structure used by real Data Availability (DA) clients.
  • Header Storage and Retrieval: The DummyDA now stores Header objects with timestamps when blobs are submitted and when the height ticker advances. A GetHeaderByHeight method has been added to allow retrieval of these stored headers.
  • Timestamp Determinism in Tests: The Retrieve method has been updated to utilize the stored timestamps from headers, ensuring consistent and deterministic timestamp behavior in test environments, aligning with real DA client interactions.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request aims to add header storage and retrieval to DummyDA to improve timestamp determinism in tests. The changes are well-structured, introducing a Header type and integrating it into blob submission, height ticking, and data retrieval. However, I've identified a couple of areas in the new and modified functions where time.Now() is used as a fallback. This undermines the goal of determinism. My review provides specific suggestions to address these points and ensure the test utility is fully deterministic, aligning with the pull request's objective.

Comment on lines 288 to 293
if height <= currentHeight && height > 0 {
return &Header{
Height: height,
Timestamp: time.Now(),
}, nil
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The goal of this PR is to introduce timestamp determinism in tests, but this logic does the opposite when a header is not found for a valid height. By creating a new header with time.Now(), any test that relies on this function will become non-deterministic, as each call can return a different timestamp for the same height.

Furthermore, the function's documentation comment says "Returns nil if no header exists for the given height", but this implementation contradicts it by generating a header on the fly.

To ensure determinism and align with the documentation, you should not generate a header here. If a header for an empty block is expected, it should have been created by the StartHeightTicker mechanism. If it's not present, it means the block doesn't exist in the test's context, so returning nil is the correct and deterministic behavior.

if height <= currentHeight && height > 0 {
	// To ensure determinism, return nil. A header for an empty block
	// should be created by the height ticker if needed.
	return nil, nil
}

@codecov
Copy link

codecov bot commented Dec 20, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.23%. Comparing base (86b01ad) to head (889f5c8).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2946      +/-   ##
==========================================
+ Coverage   59.15%   59.23%   +0.08%     
==========================================
  Files          90       90              
  Lines        8632     8632              
==========================================
+ Hits         5106     5113       +7     
+ Misses       2944     2937       -7     
  Partials      582      582              
Flag Coverage Δ
combined 59.23% <ø> (+0.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

alpe
alpe previously approved these changes Dec 22, 2025
Copy link
Contributor

@alpe alpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good start. IMHO the GetHeaderByHeight method is a bit inconsistent but I don't want to block you on this.

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@tac0turtle tac0turtle requested a review from alpe December 22, 2025 13:41
alpe
alpe previously approved these changes Dec 23, 2025
Copy link
Contributor

@alpe alpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, thanks for the update

randygrok
randygrok previously approved these changes Jan 2, 2026
@tac0turtle tac0turtle enabled auto-merge January 2, 2026 11:20
Copy link
Contributor

@randygrok randygrok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just waiting for the e2e to pass and we are good

@tac0turtle tac0turtle added this pull request to the merge queue Jan 2, 2026
Merged via the queue into main with commit 42228e2 Jan 2, 2026
27 of 28 checks passed
@tac0turtle tac0turtle deleted the feat/dummy-da-header-support branch January 2, 2026 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support header getByheader in dummy da

4 participants